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BACKGROUND OF THE INVENTION 



The present invention relates to an image 
processing apparatus and method for receiving still 
image information and outputting the information as 

10 video signals for display and, more particularly, to an 
image processing apparatus and method which digitize 
still image information, inputted from an image sensing 
unit, store the digitized information (image data) in 
frame memory, read out the image data from the frame 

15 memory, generate video signals for display, and output 
or display the video signals. 

Conventionally, there is an image processing 
apparatus which accepts still image information as input 
inf ormation, stores the still image information in frame 

20 memory, then outputs the information to a display device 
As an image processing apparatus of this type, a film 
player which senses images on a developed photograph 
film using a line sensor or a field sensor, such as CCD 
(charge coupled device) , converts the sensed images into 

25 image signals, and displays the images on a display has 
been suggested. 
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An external view of a system including such a film 
player is shown in Fig. 22. In Fig. 22, reference 
numeral 560 denotes a film player (main body) , and, 
through a film insertion opening 562 of the film player 
560, a film cartridge 561 is inserted. In the example .pf 
Fig. 22, an APS (advanced photo system) film cartridge 
is shown for providing a developed film. For providing a 
35mm film (sleeve or mount) , a member for holding a 
sleeve (or mount) is used. Further, reference numeral 
563 denotes a display device conforming to television 
standards; 5 64, a cable for transferring video signals; 
565, a cord; 566, a remote controller; and 661 to 668, 
various control buttons. 

Next, a brief view of an APS film is shown in Fig. 
23. As shown in Fig. 23, a film 1102 is preserved within 
the film cartridge 561, wound around a spool 1101. The 
film 1102 has perforations 1103 and electromagnetic 
recording areas 1104. 

A destination of the output from the film player 
560 may be the conventional display device 563 shown in 
Fig. 22. The film player 560 and the display device 563 
communicate using video signals. Since the display 
device 560 conforming to the television standards 
performs scanning in synchronization with a horizontal 
synchronizing signal and a vertical synchronizing signal 
(or a composite signal in which the foregoing two kinds 



of synchronizing signals are mixed) , and image signals 
are combined with the synchronizing signals to form the 
video signals. 

As for video signals, there are several standards; 
NTSC (National Television System Committee) which is a 
color television standard adopted in Japan and the 
United States, PAL (Phase Alteration by Link) which is 
one of color television standards, SECAM (Sequentied 
Colours Memoir) which is another one of the color 
television standards, and so on. As typical video 
signals, waveforms of NTSC (EIA RS-170) video signals in 
a four- field sequence are shown in Fig. 24. 

A user interface to the display device 563 
connected to the film player 560 is the remote 
controller 566 shown in Fig. 22, for instance. In the 
example shown in Fig. 22, the remote controller 566 is 
connected to the film player 560 via the cord 565, but 
infrared is widely used instead of the cord 565. 

In the APS, 15-, 25-, and 40-exposure films are 
available. All the images exposed on a film are printed 
out as an index print after the film is developed. A 
brief view of an index print is shown in Fig. 25. For 
displaying a desired image on a film using the film 
player 560, a user selects the exposure number on the 
basis of information which is similarly to the index 
print, using the remote controller 566. Therefore, it is 




common that the film player 560 has a function for 
displaying information similarly to the index print. 

Fig. 26A to 26C show cases where a display screen 
is divided in three different ways for displaying index 
5 images. Fig. 2 6A shows a case where an index image is... 
displayed in a manner similar to the index print. Since 
the maximum number of exposures is 40 in the APS as 
described above, if the display screen has 512 x 1024 
pixels in the vertical and horizontal directions, 
,lg 10 respectively, then the size of each thumbnail image of 

the index image is 84 x 112 pixels. However , the rati o 
^■f of the height to the width of a display screen 

conforming to the NTSC standard is 3:4, and, for 
id expressing each pixel of the index image as a square, it 

fJ3 15 is general to edit the index image so as to be expressed 

vf| within 480 x 640 pixels, in consideration of the size of 

memory for storing the edited index image. Fig. 26B 
shows a case where the index image shown in Fig. 2 6A is 
edited to be within the size of 480 x 640 pixels. 
20 Further, it is common to display the image on the 
display screen as shown in Fig. 26C. 

A user selects a desired thumbnail image from the 
index image displayed as shown in Fig. 26C. The buttons 
661 to 664 of the remote controller 566 are for changing 
25 a selected image among the thumbnail images (a selected 
image is high-lighted by being displayed in reverse or 
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surrounded by a frame, for instance) to up, down, right 
and left. After a desired thumbnail image is selected 
from the index image, the selection button 667 is 
pressed to set the selection of the thumbnail image. 
5 Further, in order to cancel the selection of the image., _ . 
~ " " the cancel button 668 is to be pressed. After the 

selection, the original image of the selected thumbnail 
image is displayed by itself on the display screen. 
Thereafter, it is possible to change images to be 
10 displayed using the up button 665 and the down button 
T: 666. Further, it is possible to configure the system to 

^ display the index image when the cancel button 668 is 

^ pushed while displaying images one by one. 

CI For displaying an image read by the conventional 

£5 15 film player on the display device adopting television 

i-.,.= 

yrt standards as described above, image signals should be 

combined with the synchronizing signals as shown in Fig. 
24. 

Fig. 27 is a block diagram showing the main 
20 components of a conventional apparatus for video display. 
The apparatus includes a controller 901, a storage unit 
902, a decoder 904, an A/D converter 905, a D/A 
converter 906, a video encoder 907, and video memory 
1201. The source of image information in the film player 
25 is a film, as described above, and images on the film 

are dealt with in a form of electric signal obtained by 
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a CCD, for instance, as a result of photoelectric 
conversion. 

Referring to Fig. 27, electric image signals (level 
signals) are digitized by the A/D converter 905, then 
processed by the decoder 904 for,. extracting, valid- image- 
signals from the digitized signals. Although the order 
of the input image signals may differ depending upon the 
scanning method of the CCD, the digitized signals of a 
frame image are eventually stored in the storage unit 
902. The stored data is transferred to the video memory 
1201, and consecutively outputted to the D/A converter 
906 at a video rate. The digital signals converted into 
analog signals by the D/A converter 906 are encoded into 
video signals which include luminance information, color 
inf ormation, and synchronizing information by the video 
encoder 907, then outputted to the display device 563 
(Fig. 22) . 

In the film player 560 as described above, the 
source of image information is a photograph film 
(developed) , not a moving image, and it is necessary to 
keep providing video signals of the selected still image 
on the film to the display device 563. Furthermore, in 
order to change the currently displayed image to another 
image, the content of the storage unit 902 must be 
updated to image information of a new image to be 
displayed next while outputting video signals of the 



current image. In the conventional apparatus, the 
storing and the reading of data are performed in 
parallel using the video memory 1202 of a dual-port type. 
In this configuration, the storage unit 903 and the 
5 video memory 1201 are used for different .purposes., .and ___ _ __ 

when generating an index image, for instance, images on 
the film are sensed and stored in the storage unit 902 
one by one, the size of each image is reduced, then the 
reduced (thumbnail) image is stored in the video memory 
10 12 01 in an index image format, thereby the index image 
as shown in Fig. 26C is outputted to the display device 
563 . 

In most cases, the output from a CCD can be 
directly used for display, as apparently seen in a case 

15 of a camcorder. When sensing a moving image, image 
information changes as time elapses, and it is not 
necessary to temporarily store the image information in 
the storage unit 902 (except a case of performing 
digital processing on the image information) . In 

20 contrast, in the film player 560 as described above, a 

frame of a film is sensed by a CCD while illuminating it 
from backside, and the illuminating of the frame, for 
scanning, for a long time causes a raise in temperature 
inside the film player 560 to a high degrees. Thus, the 

25 light is turned off after necessary information is 
stored in the storage unit 902. 
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With the foregoing reasons, video signals are 
generated using the storage unit 902 and the video 
memory 1201. However, a dual-port memory device is more 
expensive than a single-port memory device, and it is 

~ _ ..5. preferred to.avoid using, dual^port memory- devices to - - 

provide an inexpensive film player. There is an 
apparatus using a single-port memory device instead of a 
dual -port memory device; however, in this apparatus, 
m while reading data (video signals) from the single-port 

Wit? 

. 

r|s 10 memory, updating of the content of the memory is 

?*; inhibited except during synchronizing periods. In other 

words, storing of data of an image to be displayed next 
4* must be performed during the synchronizing periods. 

C? Fig. 28 shows timing of displaying a frame image 

C:3 15 for visually illustrating the video synchronizing 

yl periods. Referring to Fig. 29, a frame period is 1/30 

second, and a field period is 1/60 second (16.7 ms) . In 
NTSC, an interlacing method (a scanning method adopted 
in a raster-scanning type display device) is adopted, 
20 and images of two different fields are alternatively 

outputted. In Fig. 29, there is 10.9 |xs for a horizontal 
synchronizing period in every horizontal scanning period, 
and 571.5 |lls for a vertical synchronizing period in 
every vertical scanning period. 
25 It is understood from the aforesaid numbers that 

periods when updating of data, while reading signals 
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from a single-port memory device, used as the video 
memory 12 01, is allowed is only about 2 0% of one field 
period. Even if the memory has a capacity of storing 
images of several fields and a new image is written to 
_ _5_ -the memory- part -with part - in an- area- other than an -area - - 
where image signals are currently read out, it takes 
several field periods to store a frame image. After the 
new frame image is stored, the area, in the video memory 
m 12 01, which is accessed for reading image data is 

5| 10 changed to the area where the new image is stored. 

>.:k 

ff f Therefore, in order to optimize the user interface 

by shortening the period required since another image is 
^* requested to be displayed until the requested image is 

M actually displayed, it is preferable to use a dual-port 

M 15 memory device or a plurality of single-port memory 

ijri devices while selecting one of them each time after a 

new image is stored. 

In either case, for providing a satisfactory user 
interface, the cost of an apparatus becomes high. 
20 Furthermore, use of a plurality of memory devices makes 
the configuration of a control circuit, such as a bus 
arbiter, complicated. 

More specifically, in an image processing apparatus 
which receives still image information and outputs video 
25 signals for display, especially in an apparatus, such as 
a film player, which is required to consecutively 
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display a plurality of still images, it is necessary to 
temporality store each still image in a storage. 
Furthermore , if the storage is a single-port device, 
while reading data of the still image from the storage 

— 5- - at- -video- ra-te 7 -data -can- -be updated only during- the - 

synchronizing period; or the storage has to be a dual- 
port device. s 

The aforesaid control is for simply displaying a 
still image of an arbitrary frame. Further, there is a 

10 case where the apparatus is added with a function of 

rotating a displayed image by operating the buttons of 
the remote controller 566. More specifically, it is 
possible to configure the system so that the button 661 
is for designating rotation of an image by 90 degrees, 

15 the button 662 is for designating rotation by 180 

degrees, the button 663 is for designating rotation by 
270 degrees, and the button 664 is for designating to 
display the image in the original direction (or for 
designating to display a mirror image) , for instance. 

20 Examples of images displayed on the display device 

563 when an image is rotated as described above, are 
shown in Fig. 29A to 29D. In Fig. 29A, an image is 
originally displayed in a ratio of 3:4 (vertical : 
horizontal) . Fig. 21B shows an image when the original 

25 image, shown in Fig. 21A, is rotated by 90 degrees, Fig. 
2 9C shows an image when the original image is rotated by 
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180 degrees, and the Fig. 29D shows an image when the 
original image is rotated by 270 degrees. Note, when the 
image is rotated by 90 or 270 degrees, the size of the 
image is reduced to 75% (=3/4) of the original size so 

„ _ :_ _ 5. _ that _ the._ image _ is. _not cut because of- the size- and shape- - - — 

of the display screen. 

For performing the aforesaid rotation processing, 
including the reduction of an image, a displayed image 
is to be modified when the apparatus has the 
2g 10 configuration as shown in Fig. 27, and during modifying 

the image, no image can be displayed. In order to keep 
g *f displaying an image, the modification of the image must 

^ be completed in one vertical blanking period between 

Q fields, which requires an apparatus of very high speed. 

Q 15 Furthermore, since the reduction of an image should be 

yp performed along with the transference of a frame image 

to the video memory 1201 which is a dual -port memory 
device, as described above, higher speed and more 
complicated circuit is required. Further, in a case 
20 where storage area for storing image data which is 
currently displayed and storage area for storing 
developed (e.g., rotated) image data are identical, too, 
development of an image to be displayed next must be 
performed in one vertical blanking period so as to start 
25 displaying the developed image in the next frame period. 
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In the above cases, if DRAM is used as the video 
memory 1201, a vertical blanking period can not be 
assigned for refreshing operation, thus it is necessary 
to access all the addresses in one frame period to 
maintain the data in the DRAM._ In_ other _ words .it. is_ not- 
possible to reserve an area in the DRAM for an index 
image and keep data of the index image when displaying 
of a single image is designated, as a function of the 
film player as described above, because all the data 
stored in the DRAM has to be read in each frame period. 
Thus, it is necessary to develop an index image each 
time the index image is requested to be displayed, which 
requires a longer time than rotating an image. 

As described above, in an image processing system 
for displaying an image stored in frame memory which 
does not have video RAM for exclusive use for display, 
development of an image to be displayed next should be 
performed during the synchronizing period, or a 
plurality of memory devices or areas, which are 
independently controlled, are provided and sequentially 
used for developing images. However, in the image 
processing system, for modifying an image which is 
currently stored in the memory for display and 
displaying the modified image, there is no other way but 
developing image data in another memory area little by 



little in spare moments from reading of the data to be 
displayed (e.g., vertical synchronizing period). 



SUMMARY OF THE INVENTION 

5 

The present invention has been made in 
consideration of the above situation, and has as its 
object to provide an image processing apparatus and 
method capable of updating an image and switching to the 

10 updated image without a break in displaying of an image. 
Further, it is another object of the present 
invention to provide an image processing apparatus and 
method capable of performing processing, such as 
rotation, on an image which is currently displayed while 

15 displaying the image during the processing using general 
purpose memory, and switching to the processed image 
without a break in displaying of an image. 

Furthermore, it is another object of the present 
invention to provide an image processing apparatus and 

20 method capable of improving throughput for developing an 
image by omitting to develop blank spaces of the image. 

According to the present invention, the foregoing 
object is attained by providing an image processing 
apparatus comprising: input means for inputting image 

25 data of a still image; branching means for branching the 
image data inputted by the input means; first memory 



used for storing one of the image data branched by the 
branching means; memory controller which controls 
writing of the one of the image data, branched by the 
branching means into the first memory, into the first 

5 — memory; a-fi-rst switch- for -selecting image data--stored 

in the first memory or the other branched image data not 
stored in the first memory; and output means for 
outputting the image data selected by the first switch. 
According to the present invention, the foregoing 

10 object is also attained by providing an image processing 
apparatus comprising: first memory used for storing 
input image data; image processing means for reading the 
image data from the first memory and applying 
predetermined processing to the image data; second 

15 memory used for storing the image data applied with the 
predetermined processing; a first switch for selecting 
the image data stored in the first memory or the image 
data stored in the second memory; and output means for 
outputting the image data selected by the first switch, 

20 wherein the image processing means applies the 

predetermined processing to the image data stored in the 
first memory while the image data is also output ted to 
the output means . 

Further, according to the present invention, the 

25 foregoing object is also attained by providing an image 

processing method comprising: an input step of inputting 
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image data of a still image; a branching step of 
branching the image data inputted in the input step; a 
first storing step of storing one of the image data 
branched in the branching step; a memory control step of 
- 5- -contr-o 1-1 i-ng- timing— for- -writing- the-- one- of- the- image- data-, 
branched in the branching step, in the first storing 
step; a first selecting step of selecting image data 
stored in the first memory or the other branched image 
data not stored in the first storing step; and an output 

10 step of outputting the image data selected in the first 
selecting step. 

Furthermore, according to the present invention, 
the foregoing object is also attained by providing an 
image processing method comprising: a first storing step 

15 of storing input image data; an image processing step of 
reading the image data stored in the first storing step 
and applying predetermined processing to the image data; 
a second storing step of storing the image data applied 
with the predetermined processing; a first selecting 

20 step of selecting the image data stored in the first 
storing step or the image data stored in the second 
storing step; and an output step of outputting the image 
data selected in the first selecting step, wherein, in 
the image processing step, the predetermined processing 

25 is applied to the image data stored in the first storing 
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step while the image data is simultaneously outputted in 
the output step. 

Other features and advantages of the present 
invention will be apparent from the following 
5 description taken in conjunction with, the accompanying 
drawings, in which like reference characters designate 
the same or similar parts throughout the figures thereof 

BRIEF DESCRIPTION OF THE DRAWINGS 

10 

The accompanying drawings, which are incorporated 
in and constitute a part of the specification, 
illustrate embodiments of the invention and, together 
with the description, serve to explain the principles of 
15 the invention. 

Fig. 1 is a block diagram illustrating a basic 
configuration of an image processing apparatus according 
^ o^a f irst embodiment of^the present invention; 
FigT~'2 'is a -bTock diagram illustrating a 
20 configuration of an APS film player according to the 
first embodiment of the present invention; 

Fig. 3 is a flowchart of a control sequence of the 
APS film player shown in Fig. 2 according to the first 
embodiment of the present invention; 
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Fig. 4 is a flowchart of the control sequence of 
the APS film player shown in Fig. 2 according to the 
first embodiment • of the present invention; 

Fig. 5 is a block diagram illustrating a 
configuration of an image processing _ apparatus ^according. 



to ^a^s econd embpdiment^ the present invention; 

Fig. 6 is a flowchart of a control sequence of the 

image processing apparatus shown in Fig. 5 according to 

the second embodiment of the present invention; 
10 Fig. 7 is a flowchart of the control sequence of 

the image processing apparatus shown in Fig. 5 according 

to the second embodiment of the present invention; 
Fig. 8 is a block diagram illustrating a 

configuration of an image processing apparatus according 
15 t^^third and fourth embodiment s^o£ the present 

invention; 

Fig. 9 is an explanatory view the 656 format 
recommended by the International Telecommunication 
Union; 

20 Fig. 10 is a block diagram illustrating a 

configuration of a main portion of the image processing 
apparatus according to th e^ third e mbo diment of the 
present invention; 

Fig. 11 is a timing chart of data reduction 

25 processing according to the third embodiment of the 
present invention; 
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Fig. 12 is a block diagram illustrating a 

configuration of a data operation block for data 

reduction processing according to the third embodiment 

of the present invention; 
_5 Fig . 13 _is a block diagram .illustrating, a _ _ 

configuration of a barrel shifter used in the data 

reduction processing according to the third embodiment 

of the present invention; 

Fig. 14 is a flowchart showing image 
10 sensing/rotation processing in the image processing 

apparatus according to the third embodiment of the 

present invention ; 

Fig. 15 is a flowchart showing the image 

sensing/rotation processing in the image processing 
15 apparatus according to the third embodiment of the 

present invention; 

Fig. 16 is a flowchart showing the image 

sensing/rotation processing in the image processing 

apparatus according to th e, third e mbodiment of the 
20 present invent i on ; 

Fig. 17 is a flowchart showing the image 

sensing/rotation processing in the image processing 

apparatus according to the third embodiment of the 

present invention; 
25 Fig. 18 is a block diagram illustrating a 

configuration of a masking processing circuit of the 
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image sensing apparatus according to the fourth 

embodiment of the present invention; 

Fig. 19 is a flowchart showing timing control of an 

image sensing and developing operation in the image 
_ _ _ .5 _ processing apparatus _according_to_ the fourth embodiment. 

of the present invention; 

Fig. 20 is a flowchart showing timing control of 

the image sensing and developing operation in the image 
_^ processing apparatus according to the fourth embodiment 

j : <5 10 of the present invention ; 

~> Fig. 21 is a block diagram illustrating a brief 

%l configuration of an image processing apparatus provided 

=p with input and output interfaces; 

Q Fig. 22 is an external view of a system including a 

jp 15 film player; 

J5 Fig. 23 is an explanatory view of an APS film; 

Fig. 24 is a timing chart showing the NTSC format; 
Fig. 25 is a view showing an example of an APS 
index print; 

20 Fig. 26A is a view showing an example of an index 

image of all the images on a film; 

Fig. 26B is a view showing an example of an edited 
index image ; 

Fig. 26C is a view showing an example of a 
25 displayed index image; 
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Fig. 27 is a block diagram illustrating a main 
portion of a conventional image processing apparatus; 

Fig. 28 is a view for explaining displaying timing 
of a frame image; 

_ 5 Fig._.2?A is _a .view showing an_ e_xamp_le _ of _ a _ _ _ _ 

displayed original image; 

Fig. 29B is a view showing the image rotated by 90 
degrees ; 

Fig. 29C is a view showing the image rotated by 180 
10 degrees; and 

Fig. 29D is a view showing the image rotated by 270 
degrees . 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 



Preferred embodiments of the present invention will 
be described in detail below in accordance with the 
accompanying drawings . 

<First Embodiment> 

20 Fig. 1 is a block diagram illustrating a basic 

configuration of an image processing apparatus according 
to the first embodiment of the present invention. The 
image processing apparatus includes an A/D converter 105 
for converting input image information to digital image 

25 signals, a decoder 104 which performs processes, such as 
y correction, noise reduction, and timing adjustment, on 
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the digital image signals for extracting valid data 
(will be explained later) , a storage unit 102 for 
storing a frame of the extracted digital image signals, 
a D/A converter 106 for converting the digital image 
5 .signals into_ analog, image, signals , .a_ _video_ encoder 10-7- 
for generating video signals from the analog image 
signals, a switch 108 for selecting either the output 
from the storage unit 102 or the output from the decoder 
104, and a controller 101 for controlling the overall 
10 operation of the image processing apparatus. 

Note, data flows which is read and written to and 
from the storage unit 102 are represented by three data 
lines in Fig. 1 for an explanatory purpose; however, 
since a single-port memory device is used as the storage 
15 unit 102 in the first embodiment, data is inputted to 
and outputted from the storage unit 102 via a single 
bidirectional data line in practice. 

Next, an operation of the image processing 
apparatus having the aforesaid configuration is 
20 explained . 

In the aforesaid configuration, when writing image 
data to the storage unit 102, the switch 108 selects the 
output from the decoder 104, thereby new image data is 
inputted to the storage unit 102 while the same image 
25 data is transferred to the D/A converter 106 

simultaneously, and the image data, which is currently 
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written in the storage unit 102, is displayed on a 
display device which is connected in the downstream to 
the video encoder 107 . 

By inputting image data of one frame to the storage 
- - -- -- - - 5- - -unit 102 -as- described- above, the -image data i-n -the- - 

storage unit 102 is updated. Then, by the beginning of 
the next frame period (field sequence), the switch 108 
is changed to select the output of the storage unit 102. 
In this manner, two still images are consecutively 
fjij 10 displayed in the display device. Further, since an image 

JtS sensing system for reading an original image and 

J: providing image signals to the A/D converter 105 can be 

4 * put in a stand-by state until it is designated to read 

another image, electric power is saved. 
C3 15 Fig. 2 is a block diagram illustrating an APS film 

€3 player as an example of an apparatus using the image 

processing apparatus having the aforesaid basic 
configuration according to the first embodiment of the 
present invention. 
20 Below, the APS film player, which uses one single- 

port RAM as frame memory, capable of updating an image 
stored in the frame memory while keep displaying an 
image with a simple configuration is explained. 

Referring to Fig. 2, an image sensing device (CCD) 
25 208 scans a film, such as the film 1102 shown in Fig. 23, 
and the obtained image data is pre-processed by a 
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correlated double sampling (CDS) /automatic gain control 
(AGC) processing unit 209. Correlated double sampling is 
a technique to cancel out differences between charges 
(potentials) of respective elements of the CCD, and the 
- 5- -automatic gain- control- -is- -for automa-ti-cally--conferoll-i-ng - - 
the gain to be applied to a signal. Image information 
represented by electric signals are digitized by an A/D 
converter 210 . 

A decoder 2 03 generates digital data array (code 

10 array) having a frame format in which the digitized 

image information is synchronized. The decoder 203 also 
outputs synchronizing signals (horizontal synchronizing 
pulses (HD) , vertical synchronizing pulses (VD) , and 
pixel clock) for the frame format. In the image sensing 

15 system, a timing signal generator 211 takes the 

synchronizing signals, and generates and outputs a frame 
timing signal and a pixel timing signal to establish 
synchronization in the image sensing system. 

Further, image data is written to frame memory 202 

20 in synchronization with the synchronizing signals 

outputted by the decoder 203. The frame memory 202 is 
configured with one single-port RAM, and controlled by 
an address /status signal outputted from a memory 
controller 200. The memory controller 200 controls 

25 timing of storing valid image data on the basis of 

timing information obtained via a VD detector 204 and an 
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HD detector 2 05 for synchronizing with the image sensing 
system, an HD counter 2 06 for detecting a time when the 
valid image data starts, and an HD timer 207 for 
detecting the beginning of a valid pixel in the 

- - - - 5-. -horizontal- direction, _on the. basis. _ of the HD_and_VD . _ 

outputted from the decoder 203, 

The valid image data is data representing an image 
excluding a surrounding portion of the image, since the 
fs*s CCD 208 is usually designed to sense an area wider than 

^ 10 the image itself. Further, the valid pixel is a pixel 

^ represented by the valid image data. 

f'; 1 Data to be inputted to the frame memory 202 is the 

4* output from the decoder 203. Further, output data of the 

p frame memory 202 is transferred via the same data bus 

52 15 used for inputting the data to the frame memory 2 02, and 

43 enters a multiplexer 201; accordingly, a three-state 

buffer 219. is used to prevent data from being conflicted. 
An enable signal for the three-state buffer 219 is also 
controlled by the memory controller 200. Further, the 
20 memory controller 200 also controls selection of inputs 
in the multiplexer 201 using a switching timing signal. 
Digital image information selected by and outputted from 
the multiplexer 201 is converted into analog signals by 
a D/A converter 212, further processed by a video 
25 encoder 213, which is in the downstream to the D/A 

converter 212, and changed into video signals carrying 
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information of luminance, color, and synchronization. 
Thereafter, the video signals are outputted to a display- 
device . 

Clock timing of the D/A converter 212 and clock 

_5_ „timing_ of __the memory controller 200 are controlled in- - 
accordance with a phase-locked clock of a clock 
outputted from the decoder 203 in a phase lock loop 
(PLL) 214. Accordingly, either in a case where the 
output from the frame memory 202 or in a case where the 

10 output from the decoder 2 03 is selected and outputted 
from the multiplexer 201, a displayed image does not 
become out of synchronization. 

A timing signal for the video signals outputted 
from the video encoder 213 is generated by a composite 

15 synchronizing signal generator 215. The generator 215 
also generates a horizontal synchronizing signal H_sync 
and a vertical synchronizing signal V_sync in addition 
to a composite synchronizing signal which is the timing 
signal for the video signals. Accordingly, it is 

20 possible for the output of the video encoder 213 to 
correspond to the S terminal output (to output a 
luminance signal and a chrominance signal separately) . 
The memory controller 200 and the decoder 203 receive 
control commands from MPU 216 by command reception 

25 buffers, and execute the commands in synchronization 
with timing of a frame of an image. 
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Reference numeral 217 denotes ROM where a control 
program, to be executed by the MPU 216, including a 
control sequence shown in Figs. 3 and 4, which will be 
explained later, configured as a storage medium of the 

-present -invention As - for -the storage medium-,- - 

semiconductor memory, an optical disk, a magneto-optical 
disk, and a magnetic medium may be used, for instance. 
Further, reference numeral 218 denotes work RAM for 
temporarily storing results of operations executed by 
the MPU 216 . 

Next, the control sequence of the APS film player 
having the aforesaid configuration are explained with 
reference to Figs. 3 and 4. After the control sequence 
starts, the frame memory 202 is initialized in step S401 
This process may be skipped when it is not necessary. 
Thereafter, an image sensing system starts to operate to 
read a film image in step S402 . Note, the image sensing 
system indicates a group of units which are synchronized 
in accordance with synchronizing signals generated by 
the timing signal generator 211, namely, the CCD 208, 
the CDS/AGC 209, and the A/D converter 210. The CCD 208 
may be left functioning at all the time. The operation 
performed in step S402 is to illuminate a film image by 
a back light and read the film image so that the read 
film image can be captured (i.e., stored) by the frame 
memory 202 . 




When the read film image is ready to be captured, 
the multiplexer 201 selects the output from the decoder 
203, namely, selects image signals of the film image 
which is currently being read, in step S403. At this 
" - -- - -- -5 - time;- the decoder --2 03 -is- optimized- by the _MP_U_2L6__f or _ 

capturing the film image. Then, in step S404, the HD 
counter 2 06 is initialized for generating a timing 
signal used to capture the film image. Thereafter, the 
process waits until a command for capturing an image 

H ' 2 

10 (image capture command) is issued in step S405 . In this 
4*. stage, the film image is displayed on the display device. 

HI The image capture command is generated when a user 

43 inputs the command by operating a remote controller, for 

£3 instance, while monitoring the image displayed on the 

15 display device. Further, there may be a case where the 
>5 MPU 216 automatically issues the image capture command 

^ in a sequence of presentation. 

Further, while the process waits for the image 
capture command, the image captured in the previous 
20 image capture operation may be displayed (i.e., the 

output of the frame memory 202 is selected) (YES in step 
S406) . However, since no image data exists in the frame 
memory 202 soon after this processing is initiated, more 
specifically, until the first image is captured by the 
25 frame memory 202, a predetermined value is often written 
and developed in the frame memory 2 02 when initializing 
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the frame memory 202 in step S4 01 so that a color pack 
is displayed when the multiplexer 2 01 switches to select 
the output from the frame memory 202 due to af f irmation 
in step S406. 

5 - When an image -capture -command -is. -issued in . the . step _ 

S405, in step S407, the VD detector 204 detects the VD 
signal which indicates the start timing of the next 
frame. The VD signal is a timing pulse indicative of the 
start timing of a vertical return period, and, by 

10 detecting the timing pulse, start and end of the 

vertical return period can be determined. When the end 
of the vertical return period is detected, the HD 
counter 2 06 and the HD timer 207 are initialized in step 
S408, then wait for the HD signal in step S409. Note, 

15 the initialization of the HD counter 206 and the HD 
timer 207 may be performed while waiting for the VD 
signal in step S407. 

The HD signal is a pulse synchronized with a 
horizontal return period, which is generated for each 

20 horizontal scanning line, and the start time of each 
horizontal scanning operation is determined from the 
pulse. As the HD detector 205 detects the HD signal, the 
HD counter 2 06 and the HD timer 207 start operating in 
step S410. Next in step S411, the value of the HD 

25 counter 2 06 is compared to a predetermined value to 

determine whether or not the current row is in the range 
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of the image. In a practical frame (field) configuration, 
image information starts after several horizontal 
scanning lines are passed since the end of the vertical 
return period. Accordingly, it is necessary to set the 
--5 - predetermined- value- in consideration, of the af oresaid 
fact. In this case, the predetermined value is 
determined on the basis of the characteristics of the 
CCD 208. 

^ If the current row is not within the image range 

^ 10 (NO in step S411) , the HD timer 207 is cleared in step 

S412 and the process waits for the next detection of the 
yi HD signal in step S409. If the current row is in the 

=P image range (YES in step S411) , then whether or not a 

C3 current pixel is a valid pixel, which carries image 

£;1 15 information, in the horizontal scanning line is 

yH determined in step S413 . The process of step S413 is 

performed since there are several pixels before a pixel 
indicative of image information starts after the end of 
the horizontal return period. Further, in a case of 
20 discarding pixels so as to fit the size of the read 
image to the capacity of the frame memory 2 02, the 
position of the valid pixel should be determined in 
consideration of the number of pixels to be discarded. 
Until the pixel position reaches the position of the 
25 valid pixel, the HD timer 207 counts the number of 
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clocks, which corresponds to the number of pixels, in 
step S414. 

After the valid pixel is detected, output data from 
the A/D converter 210 is written into the frame memory 

5- -20-2 -via the - decoder- 203 - in -steps S415- and- -S416 . -In- the. . 

first embodiment, the read image, i.e., the same output 
data from the decoder 203, is selected by the 
multiplexer 212 and displayed in the display device 

^ while writing the output data to the frame memory 202. 

^ 10 Accordingly, images are displayed without a break, even 

==? during reading and writing a new image to the frame 

%J memory 202. 

4* By directly displaying the read image for a frame 

Q period while simultaneously writing the same image to 

m 15 the frame memory 2 02, the content of the frame memory 

j£ 202 is updated. Then, when the scanning of the current 

frame is completed (i.e., when the ends of an odd field 
and an even field are detected by the memory controller 
200) and a vertical synchronizing period starts in step 
20 S417, then the image sensing system is stopped in step 
S418. In the process in step S418, the back light is 
turned off, and peripheral devices are disabled, 
although the CCD 208 may be left functioning; namely, 
output data from the CCD 2 08 is invalidated in step S418 
25 Then, in step S419, the multiplexer 201 switches to 

select the output data from the frame memory 2 02. At 
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this time, the three-state buffer 219 is disabled to 
make the data bus from the decoder 203 to the frame 
memory 2 02 to high impedance. 

The V_sync signal and the H_sync signal used for 

-5 - -synchronizing the display__device are .generated by the 

composite synchronizing signal generator 215 at the same 
time the generator 215 generates the composite 
synchronizing signal, and inputted to the memory 
controller 2 00 where timing of vertical synchronization 

10 and horizontal synchronization is utilized. When the 

start of the next frame is detected by detecting the end 
of the current vertical synchronizing period in step 
S420, data stored in the frame memory 202 starts being 
read in step S421. The memory controller 200 outputs an 

15 address/status signal in synchronization with the H_sync 
signal so that corresponding pixel data is read. 

The memory controller 200 determines the end of the 
current field by detecting the start of the next 
vertical synchronizing period in step S422, and controls 

20 to switch the selection of the outputs by the 

multiplexer 201 when required. In step S423, if a 
control command instructing the display of an image 
being sensed by the CCD 208 is issued by the MPU 216, 
then the process returns to step S402 . Further, if a 

25 command to stop displaying the image is issued (YES in 
step S424) (e.g., in order to display a menu image 
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composed of super imposition and color background 
prepared independently) , output of the image from the 
frame memory 2 02 is stopped. If no command to stop 
displaying the image is issued (NO in step S424) , the 
5 process returns' to" step S420 "and" ~the "image "output ted ~ - ~ ------- 

from the frame memory 202 is kept displayed. 

For outputting color background, it is possible to 
control the video encoder 213 to output a fixed value, 
or to provide a video switch (not shown) which switches 

10 between the output from the video encoder 213 and a 

fixed value. Regarding an on-screen display device (not 
shown) used for superimposition, it may be provided 
between the D/A converter 212 and the video encoder 213, 
or downstream from the video encoder 213. 

15 According to the first embodiment as described 

above, in the image processing apparatus, which is for 
outputting video signals to be displayed on the basis of 
input still image information, having the A/D converter 

105 for obtaining digital image signals, the decoder 104 
20 for extracting effective data out of the digital image 

signals, the storage unit 102 for storing one frame of 
the extracted digital image signals, the D/A converter 

106 for converting the digital image signals into analog 
image signals, and the video encoder 107 for generating 

25 video signals from the analog image signals, the switch 
108 for selecting either the output from the storage 
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unit 102 or the output from the decoder 104, and the 
controller 101 for controlling the overall flow of data 
as well as controlling the switch 108 are provided. 
Accordingly, the content of the frame memory 202 is 
- - - - -5- updated by -a new -image without .considering, video. . 

synchronizing periods as well as without a break in 
displaying of an image. 

Further, with the aforesaid configuration and 
^ operation, it is possible to obtain the same effect 

^1 10 realized by the storage unit 902 and the video memory 

42 1201, shown in Fig. 27, of the prior art, using one 

Si single-port memory. Accordingly, the image processing 

=p apparatus can be made at a low cost. Further, it is 

Q possible to update the content of the storage unit 102 

g 15 while displaying an image which is written into the 

]K storage unit 102, without being limited to the video 

^ synchronizing periods, thus different from the prior 

art; therefore, a displayed image is changed without 
keeping a user, who instructed to change the images, 
20 waiting, thus it is possible to provide a user interface 
of the apparatus in the best condition. 

Although the aforesaid image processing apparatus 
may be configured using one single-port RAM, there are 
cases where a plurality of single-port RAMs are 
25 necessary for storing a plurality of images and 

performing various processes on the images (e.g., a case 
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of consecutively displaying a plurality of images as in 
a slide show) . In such cases, updating of an image in 
each RAM is performed in the same processing sequence as 
described above, and the obtained advantages are also 
-5- -same- as -above. — By- independently control-ling- a- plural-ity - 
of single-port RAMs, a circuit can be simplified 
comparing to a case of using dual -port memory. 

<Second Embodiment> 
10 Next, an image processing apparatus, e.g. , a film 

player, according to the second embodiment is explained 
with reference to Fig. 5. 

In the second embodiment, a method of generating an 
index image while displaying it using single-port memory 
15 is explained. The index image includes a plurality of 
thumbnail images read from the film 1102 (Fig. 23) 
arranged in the format as shown in Fig. 26C. For 
generating the index image, images read from the film 
1102 are reduced, one by one, to generate thumbnail 
20 images, and arranged sequentially in memory as each 
thumbnail image is generated. 

For displaying the index image while generating and 
arranging the thumbnail images, a plurality of single- 
port memories are necessary. Even so, since the single- 
25 port memories can be controlled independently, the 

configuration of the apparatus is greatly simplified 
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comparing to a case of using video RAM or dual-port 
memories for said exclusive use. For the explanatory 
sake, a method of generating each thumbnail image while 
temporarily displaying the full original image stored in 
5 "~ one'singTe-port memory, then - displaying an -index -image- - 
after all the thumbnail images are arranged in 
respective areas of the index image is explained in the 
second embodiment . 

The units and elements in Fig. 5, except a low-pass 

10 filter (LPF) 301, a thinning unit 302, a buffer, 303, 
and a three-state buffer 304, are the same as those 
shown in Fig. 2; therefore, explanation of them are 
omitted. The LPF 3 01, the thinning unit 3 02, the buffer 
303, and the three-state buffer 304 are for generating a 

15 thumbnail image, thus, when outputting a full frame 

image (i.e., the image is not to be reduced), the LPF 
301 and the thinning unit 3 02 are disconnected by 
switching between the thinning unit 3 02 and the three- 
state buffer 219. At this time, the three-state buffer 

20 304 provided between data bus of the frame memory 2 02 
and the buffer 3 03 should be disabled. 

The reduction processing is to reduce the number of 
pixels of an image, and equivalent to lowering the 
sampling frequency. Therefore, in order to prevent 

25 aliening distortion, it is necessary to limit the 
frequency range allowable at the current sampling 
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frequency. The LPF 301 is a filter for this purpose. By 
reducing data which passed the LPF 3 01, an image is 
reduced. The thinning unit 302 is for reducing the image, 
and, when a displayed reduced image is not expected to 
"5 have high' quality, -pixels- may- be -simply thinned -out ,- - 
otherwise, operation between adjoining pixels may be 
added to the thinning out operation. 

The buffer 3 03 is for temporally storing data, a 
part of the content in the frame memory 202, to be 
K 10 processed by the LPF 301 and the thinning unit 302. 

f " Generally, any available DRAM may be used as the frame 

^ memory. If the capacity of the RAM is 16 Mbits, for 

P instance, three frames of 640 x 480 square pixels (eight 

13 bits each for color difference signals and a luminance 

3. 15 signal of one pixel) can be stored. The RAM is divided 

(f into partitions, and image data which is currently read 

from the film 1102 is stored in one of the partitions, a 
part of the image data is down-loaded to the buffer 303 
and reduced, then the obtained thumbnail image is stored 
20 in another partition of the RAM. By reducing desired 
film images into thumbnail images and storing them in 
another partition, then reading out image data from the 
other partition and outputting it to a display device, 
an index image is displayed. 
25 Figs. 6 and 7 are flowcharts showing a control 

sequence of the image processing apparatus having the 
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aforesaid configuration according to the second 
embodiment, and program codes corresponding to 
respective steps are stored in the ROM 217. In Fig. 6, 
the same processes as those shown in Fig. 3 are referred 
to by the same step numbers, "and explanation of them" are- 
omitted. 

It is assumed that, by the process of step S515, a 
user has selected whether or not to display an index 
image using a remote controller, and the MPU 216 has 
issued a control command to the memory controller 200. 
Both in a case of displaying a single image and a case 
of displaying an index image, processes from sensing an 
image for preparing to write the sensed image to the 
frame memory 202 are the same. The frame memory 2 02 is 
divided into a plurality of partitions, as described 
above. One of the partitions is assigned to store a 
frame image. After a frame image is stored in the 
partition (YES in step S516) , in a case where the 
control command from the MPU 216 is not to instruct to 
display an index image (NO in step S517), the process 
proceeds to step S521. 

If the control command from the MPU 216 is to 
instruct to display an index image (YES in step S517), 
then an area where a thumbnail image is to be stored is 
designated in step S518, and a thumbnail image is 
generated and stored using the units 301 to 304 in step 
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S519. The area designated in step S518 is where in a 
frame area (e.g., in 640 x 480 pixels) the thumbnail 
image is to be arranged. In the index image generating 
processing, until all the desired thumbnail images are 
5 arranged ~~(e".g.~, "until the area, :~ having a maximum - - ----- 

capacity of storing a frame image, reserved for storing 
an index image becomes full) , the processes of step S504 
to S520 are repeated. 
«^ When the desired thumbnail images are stored, it 

2* 10 should be in a vertical synchronizing period for 

!** scanning by the CCD 208 and scanning the display. During 

^ this period, the image sensing system is stopped, and an 

4« image stored in the frame memory 202 starts being 

M displayed in the next frame (field) period. The concept 

Q 15 of stopping the image sensing system is the same as that 

y| explained in the first embodiment. Further, when the 

process moves from the step S40 6 to step S521, the image 
sensing system is stopped during the vertical 
synchronizing period. 
20 In step S522, the multiplexer 210 switches the 

selection from the output of the decoder 203 to the 
output from the frame memory 2 02. Thereafter, the 
process waits for the end of the vertical synchronizing 
period in step S523 . After the vertical synchronizing 
25 period ends and the next field period for scanning the 
display device starts, the data read from the frame 
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memory 202 is converted into video signals by the video 
encoder 213. In the second embodiment, data is read for 
one horizontal scanning line in step S524, and the 
reading of the data is repeated until the next vertical 
.synchronizing jperiod .starts .(steps S525. . and_.S5.2-6-) - -Note, 
although it is not shown in Fig. 4, the processing as 
described above is also performed in the first 
embodiment. After the next vertical synchronizing period 
starts, the image to be displayed in the next frame 
period is checked. If the image to be displayed in the 
next frame period is a new image read by the CCD 208 
(YES in step S527) , then the process returns to step 
S402 and the image sensing system starts being operated. 
If it is requested to display a menu image (YES in step 
S528) , then the process proceeds to another processing 
of displaying a menu image. Otherwise, it is determined 
that the image currently displayed is to be displayed in 
the next frame period, and the process returns to step 
S523 and the processes of steps S523 to S528 are 
repeated. 

According to the second embodiment as described 
above, in addition to the same effects realized by the 
first embodiment, it is possible to generate an index 
image by dividing a memory into a plurality of areas, 
storing an input image to one of those areas, reducing 



the image, and storing the reduced image into another 
area of the memory. 

<Third Embodiment > 
_ _5_ _ . _ . Next,, the_ third_ embodiment .of . the present^ invention 
will be explained in detail with reference to Figs. 8 to 
17. 

In the third embodiment, a case of rotating an 
image as shown in Fig. 29 will be explained. In 

10 switching to display a rotated image, change over from a 
currently displayed image to the rotated image can be 
accomplished smoothly by changing the images at a start 
timing of a new frame (or a new field) . In the third 
embodiment of the present invention, an image processing 

15 apparatus capable of changing a displayed image to a 

rotated image smoothly as well as developing a rotated 
image to be displayed next while displaying an original 
image using a general-purpose memory, without a video 
RAM for exclusive purpose is explained. 

20 Fig. 8 is a block diagram illustrating a 

configuration of an image processing apparatus according 
to the third embodiment of the present invention. The 
image processing apparatus includes path selectors 2100a 
to 2100f, a controller 2101, a modification block 2103, 

25 a decoder 2104, an A/D converter 2105, a D/A converter 

2106, a video encoder 2107, first frame memory 2021, and 
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second frame memory 2022. Further, reference numeral 
2108 denotes a switch for controlling the respective 
path selectors 2100a to 2100f; 2109, an image sensing 
device, such as a CCD; and 2110, a display device. 
"5~ -- Referring -to- F-ig. -1 the - image - sensing - device -2-109 - 

receives light from a still image and converts the 
received light into electric signals. The A/D converter 
2105 converts the electric signals inputted from the 

m image sensing device 2109 into digital image information. 

10 The decoder 2104 performs various processes for forming 
video signals (video signal processing) on the output 

;^ from the A/D converter 2105. The controller 2101 

performs settings of the decoder 2104, status control 

C:3 (not shown) of the first and second frame memories 2021 

rj 15 and 2 022, and control of the path selectors 2100a to 

2100f as will be described in flowcharts shown in Figs. 

^ 10 to 13. The D/A converter 2106 converts digital data 

into analog data to be displayed, and the video encoder 
2107 outputs video signals having a format of, e.g. , 
20 NTSC composite signals. 

The first and second frame memories 2021 and 2022 
are two different memory areas which are controlled 
separately. To each of the first and second frame 
memories 2 021 and 2022, the path selectors 2100d and 
25 2100e for independently outputting data and returning 

the data into the respective memories are connected. The 
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modification block 2103 is provided between the path 
selectors 2100d and 2100e, which develops an image. The 
path selectors 2100b and 2100c select either new image 
data which is obtained by converting input image 

~5 " information into digital image- data by the -A/D converter- -- - - - 

2105 and applying video signal processing to the digital 
image data by the decoder 2104, or developed data via 
the modification block 2103, and input the selected data 
^ to the first and second frame memories 2021 and 2022, 

iff* 10 respectively. The display device 2110 displays output 

+; data for display. Note, the display device 2110 may be 

y* included in the image processing apparatus or may be 

4* provided outside of the image processing apparatus. 

C3 The path selector 2100a selects either the first 

p 15 frame memory 2021 or the second frame memory 2021 for 

J inputting new image data. The path selector 2100f 

selects either the output from the frame memory 2021 or 
the output from the frame memory 2 022 to be inputted to 
the D/A converter 2106 as a source used for generating 
20 data to be displayed. The selected output is converted 
into analog data by the D/A converter 2106, then formed 
into video signals having a format of, e.g., NTSC 
composite signals, by the video encoder 2107. 

Note that, in Fig. 8, two memories for storing 
25 images, namely, the first and second frame memories 2021 
and 2022, which can be controlled independently are 
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used; however, the number of memories is not limited to 
two, and more than two memories of this kind may be 
provided. Further, if the manufacture cost is not 
considered, dual -port memory may be used. However, for 

- 5- -reducing- the- maauf ac-tu-re- cost, -DRAM -is - preferably .used 

as the first and second frame memories 2021 and 2022. 
Further, if noises in lower bits, which can not be 
removed by the decoder 2104, are added to input data 
„^ when the input data is converted to digital data, a 

2>J 10 circulating- type noise reduction mechanism is necessary. 

4* In such a case, if data is to be inputted to and 

>! outputted from memory at a multiplicity of a sampling 

45 rate, for instance, it is best to use synchronous 

ri dynamic RAM (SDRAM) . 

p 15 Further, in the configuration as shown in Fig. 8, 

JK image data is converted to analog data by the D/A 

converter 2106, then encoded to video signals (analog) 
by the video encoder 2107; however, it is possible to 
output image data in a digital component video signal 
20 format as shown in Fig. 9, and a decoder for exclusive 

use for decoding the data in the aforesaid format may be 
provided. Fig. 9 shows the 656 format recommended by the 
International Telecommunication Union (ITU, previous 
CCIR) , and more specifically, Interlaces for Digital 
25 Component Video Signals in 52 5 -Line and 62 5 -Line 
Television System Operating at the 4:2:2 Level of 



-43- 



Recommendation 601. Thus, the present invention is 
applicable to comply with the Recommendation 601. 
Further, the present invention is applicable to any 
image data format for display regardless of the format, 

~~ - 5 digital- -or- analog-.- ----- _ _ . 

Fig. 10 is a block diagram illustrating a 
configuration of a unit, between the decoder 2104 and 
the D/A converter 2106, shown Fig. 8, of the image 

^ processing apparatus, featuring the third embodiment of 

'0. 10 the present invention. 

42 Referring to Fig. 10, the unit includes ' first SDRAM 

Si 2201, second SDRAM 2200, a data selector 2202, a first 

4: dual-port buffer 2203, a second dual -port buffer 2204, a 

rg third dual -port buffer 2205, a buffer manager 2206, an 

15 SDRAM controller 2207, a state manager 2208, a command 
5j manager 2209, a mode manager 2210, a timing observer 

^ 2211, a flag detector 2212 for controlling timing of 

frames, a barrel shifter 2213, switches 2o, 2p and 2q, 
and three-state buffers 2r and 2s. 
20 More specifically, the controller 2101, .shown in 

Fig. 8 which includes the configuration shown in Fig. 10, 
receives a control instruction and setting data from MPU 
(not shown) , and performs input/output control of 
memories based on a predetermined sequence in accordance 
25 with the received instruction and data. Further, in the 
third embodiment of the present invention, the first and 
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second frame memories shown in Fig. 8 are configured 
with the first and second SDRAMs 2201 and 2200, 
respectively, as shown in Fig. 10, Further, the 
modification block 2103, shown in Fig. 8, for developing 

5 ~ an- image -and -the -controller 2101 -are configured -with 

hardware, i.e., buffers and logic circuits arranged in a 
single application specific IC (ASIC) , in the third 
embodiment . 

The data selector 2202 is also arranged in the 
yp; 10 ASIC; accordingly, the data selector 2202, the first to 

4* third dual-port buffers 2203 to 2205, the buffer manager 

Si 2206, the SDRAM controller 207, the state manager 208, 

=p command manager 209, the mode manager 210, the 

p timing observer 2211, the flag detector 2212, the barrel 

p5 15 shifter 2213, and the switches 2o to 2q are configured 

5 in one-chip ASIC. Below, correspondence between units 

Hl and elements shown in Figs. 8 and 10 are described. 

As described above, Fig. 10 shows a specific 
example of the path selectors 2100a to 2100f , the 
20 controller 2101, the modification block 2103, and the 
first and second frame memories 2021 and 2 022 shown in 
Fig. 8. The data selector 2202 is a. unit including the 
path selectors 2100a to 2100 f collectively, and 
controlled by the mode manager 2210 which is a 
25 configuration element of the controller 2101. 
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In Fig. 8, the output from the first or second 
frame memories 2021 or 2022 is directly inputted to the 
D/A converter 2106 via the path selector 2100f; however, 
when SDRAM is used as frame memory as shown in Fig. 10, 

~5 sirice'it is "necessary- to control- a- burst- transfer- - 

sequence, RAS (Row Address Strob) pre-charging, 
refreshing, and so on, the output is temporarily stored 
in a buffer and time matched (stuffing) . Therefore, in 
Fig. 10, the dual-port buffer 22 05 for adjusting timing 

10 for the video encoder 2107 is added. Further, if the 
video encoder 2107 conforms to a digital I/F, then a 
unit for forming the output into signals having the 
digital component video signal format, is needed in 
place of the D/A converter 2106. 

15 The first to third dual-port buffers 2203 to 2205 

are not independent devices differing from video RAM, 
but implemented as microcells in ASIC. Therefore, 
comparing to a case of using independent dual-port RAM 
as frame memory, using SDRAM and ASIC of a sufficient 

20 chip size reduces the cost of the image processing 
apparatus . 

Further, each of the first to third dual-port 
buffers 2203 to 22 05 does not have the capacity of 
storing an image frame, but has the capacity of storing 
25 data of (SDRAM data width) x 256, at most. Regarding the 
first and second dual-port buffers 2203 and 2204, 
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single-port buffers may replace them by providing an 
arbiter having a control function for avoiding data 
conflict; however, throughput for processing a single 
frame of image data decreases. 

The first" arid " second" "dual -port buffers- 2-203 -and - - - 
2204, the switches 2o to 2q, the three-state buffers 2r 
and 2s, and the barrel shifter 2213 are configuration 
elements of the modification block 2103 shown in Fig. 8. 
The buffer manager 2206, the SDRAM controller 2207, the 
state manager 2208, the command manager 2209, the mode 
manager 2210, the timing observer 2211, and the flag 
detector 2212 are configuration elements of the 
controller 2101 shown in Fig. 8. In Fig. 10, lines, 
which connect blocks, having arrowheads are for 
transferring status control signals including data 
setting and a control trigger signal, and lines, which 
also connect blocks, with slashes indicate data bus for 
transmitting image data. Arrows near the lines with 
slashes (i.e., data buses) show the directions of data 
flow. 

Next, operation of each element shown in Fig. 10 is 
explained. As described above, the feature of the third 
embodiment of the present invention is that the content 
of frame memory (image data) which is currently stored 
is displayed, and, when modification (e.g., rotation) of 
the content of the frame memory is requested, while 




continually displaying the content of the frame memory, 
the identical image data is copied to another frame 
memory which is independently controlled, and the copied 
image data is modified. Then, after the modification of 
5 " the "image "data " (preparation -of- -data- -to-be- ou-tputted) - is--- 
completed, the displayed image is changed to the 
modified image at the start of the next frame period. 

The first and second SDRAMs 2201 and 2200 
correspond to the aforesaid plurality of frame memories 

10 which are independently controlled. The statuses of the 
first and second SDRAMs 22 01 and 2200 are controlled by 
the SDRAM controller 2207. In response to the control by 
the SDRAM controller 2207, data is written to and read 
from the first and second SDRAMs 2201 and 2200. The 

15 SDRAM controller 2207 controls the command statuses and 
addresses of the first and second SDRAMs 2201 and 22 00 
separately. Further, the SDRAM controller 22 07 also has 
to control the first and second SDRAMs 2201 and 2200 in 
synchronization with the first to third dual-port 

20 buffers 2203 to 2205, which the data inputted to or 

outputted from the first or second SDRAM 2201 or 2200 is 
to be transferred from or to, on the basis of data 
input/output status of the first to third dual-port 
buffers 2203 to 2205. Accordingly, the controller 2207 

25 outputs synchronizing signals to the buffer manager 22 06 
for managing the status of each buffer. 
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The operation of the buffer manager 22 06 follows 
the operation of the SDRAM controller 2207 , and the 
SDRAM controller 22 07 generates a SDRAM status signal at 
each clock timing in response to a state control timing 
5 signal from the~~s t a te " manager 2208 \™ Different- -data - — -- 
input/output control of frame memory is performed when 
capturing a read image, when developing an image, e.g., 
for rotation, and when generating an index image, for 
p instance. In this stage, conditions for selecting one of 

S=1 10 several sequences of data input/output control for the 

3 respective operations (mode) are designated by the 

command manager 22 09. 
^ The mode manager 2210 receives a memory control 

|3 instruction from MPU, instructs assigned functions 

Q 15 (input/output statuses) of the first and second SDRAMs 

S : :S 

2201 and 22 00 to the data selector 2202, and sets a 
command array necessary for a designated mode to the 
command manager 2209. The command manager 2209 sets a 
data input/output control sequence to be performed in 

20 the next frame period in the SDRAM controller 2207 in 
accordance with the command array. Further, it also 
notifies the state manager 2208 of the selection result 
of the data input/output control sequence for the SDRAMs. 
The state manager 2208 designates timing to output a 

25 status signal to the SDRAM controller 22 07 on the basis 
of timing information of frame period transferred from 
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the timing observer 2211, as well as switches states of 
input data for the second dual -port buffer 22 04 (three- 
state buffer 2r) . The foregoing operation is performed 
to avoid conflict of data on the bus, since the bus 
5 between the second "diiaT^port buf f er -2204- is for bi=^ _ _ _ 
directional communication . 

The bi-directional control of data to/ from the 
second dual-port buffer 2204 is necessary for performing 
reduction of an image when rotating the image by, e.g., 

10 90 degrees. In the third embodiment, the reduction of 
the image is performed in the horizontal and vertical 
directions in time division. Thus, in a case of 
performing reduction of the image in the horizontal 
direction first, after data is pooled in the first dual- 

15 port buffer 2203, reduction of the image in the 

horizontal direction is performed by the barrel shifter 
2213, and the obtained data is temporarily stored in the 
second dual-port buffer 2204. Thereafter, data is read 
from the second dual-port buffer 2204^ for reducing the 

20 image in the vertical direction, the switch 2p is 

changed so as to store the data to the second dual-port 
buffer 2204 via the barrel shifter 2213. The three-state 
buffer 2s is used for securing a path for storing the 
reduced image data in the second dual -port buffer 2204, 

25 and read address and write address of the three-state 

buffer 2s should be controlled separately. In a case of 
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rotating an image by 180 degrees, the image does not to 
be reduced; therefore, the switch 2q is controlled so 
that the data does not select the output from the barrel 
shifter 2213, Further, when rotating an image, the 

5 switch" 26 is~c"6ntrolled to -select - the input from the. 

switch 2q. 

As shown in Fig. 29B or 29D, for rotating an 
original image by 90 or 270 degrees, the image should be 
m reduced so that width of the original image is displayed 

^ 10 within the height of a display screen whose the aspect 

ratio is 3:4 (height : width) . If the image is not reduced, 
there would be an area or areas of the original image 
HP which are not displayed since the height of the rotated 

C3 image is greater than the height of the display screen. 

5;l 15 In addition, there would be an area or areas in the 

=S display screen where no image is displayed. Accordingly, 

in order to display the overall image, if the aspect 
ratio of the original image is 3:4, it is necessary to 
reduce the size of the image to 7 5% both in width and 
20 height . 

Next, the reduction operation associated with the 
rotation of an image is explained with reference to Figs. 
11 to 13. As described above, in order to display an 
overall image after rotating the image by 90 or 27 0 
25 degrees, since the aspect ratio of the image is 3:4, a 
reduction operation for reducing the size of the image 
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to 75% is required. The barrel shifter 2213 shown in Fig. 
10 is for performing the reduction operation. Fig. 11 
shows a time chart for operating data to reduce an image 
by the barrel shifter 2213. Referring to Fig. 11, in 
5 accordance "with "a" system "clock, data is read - - ------ ^ _ 

sequentially from the first dual-port buffer 2203, and 
delayed by one clock period to generate data A. Non- 
delayed data is referred to as data B. Coefficients to 
m be used for multiplying the data A and data B are 

flj 10 coefficient sequences, coff A and coff B, respectively. 

j!l The data A and data B are multiplied by the coefficient 

J: sequences coff A and coff B, respectively, and the 

corresponding products are added. The sum is referred to 
;3 as "result". As shown in Fig. 11, data is discarded 

M 15 every fourth pixel and data for three pixels remains, 

J| thereby the size of the image is reduced to 75%. In the 

third embodiment, the reduction operation is performed 
in the horizontal direction and in the vertical 
direction in time division using a single circuit; 
20 however, the present invention is not limited to this, 

and two such circuits may be provided for the respective 
directions . 

Fig. 12 is a block diagram illustrating details of 
the first dual-port buffer 2203 and the barrel shifter 
25 2213 as a block for operating data. For accomplishing 
the third embodiment, function for delaying the read. 
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data for one clock period to obtain the data A is 
required in the barrel shifter 2213. In the example 
shown in Fig. 12, the function is realized by a flip- 
flop for each bit of the data bus. The coefficient 
5 " sequences" doff "A' and'~caff" B- are set in- an arithmetic-- .. 
block shown in Fig. 12. 

The multiplied results between the respective 
coefficients of coff A and coff B and the data A and 
data B, are obtained using shift registers by shifting 

10 the data by one bit and two bits, without using a 

divider. In the third embodiment, since it is necessary 
to down load data representing all the pixels of a frame 
image from memory, the barrel shifter is used to 
complete the shifts in one clock period to improve 

15 performance. 

Fig. 13 shows a detailed configuration of the 
barrel shifter 2213 for illustrating the details of the 
arithmetic block. A barrel shifter is capable of 
arbitrarily exchanging the bit order of input data. In 

20 the third embodiment, data is shifted toward the least 

significant bit to realize a multiplication of l/2 n (n is 
natural numbers), and two barrel shifters, a barrel 
shifter A 2131 for multiplying the data A by the 
coefficients coff A, and a barrel shifter B 2132 for 

25 multiplying the data B by the coefficients coff B, are 
provided. Each of the barrel shifter A 2131 and the 
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barrel shifter B 2132 has two shift registers; one is 
set to shift data by one bit, and the other is set to 
shift the data by two bits. The shift amount may be 
arbitrarily set for realizing another coefficient from a 

5 combination* (gate) circuit." The" coefficients- to- be -- 

multiplied are controlled as a sequence for each clock 
by the shift manager 2208. With the above configuration 
of the barrel shifters A 2131 and B 2132, for 
multiplying data A or B by a coefficient 0.25, only the 

10 data shifted by two bits is outputted, and for 

multiplying data A or B by a coefficient 0.5, only the 
data shifted by one bit is outputted. Further, for 
multiplying data A or B by a coefficient 0.75, the data 
shifted by one bit (xO.5) and the data shifted by two 

15 bits (X0.25) is added. Therefore, each of the barrel 
shifter A 2131 and the barrel shifter B 2132 has an 
adder for adding products (i.e., one-bit-shifted data 
and two-bit-shifted data) outputted from the two shift 
registers. In Fig. 13, reference numeral 2133 denotes 

20 the adder for adding the outputs from the barrel shifter 
A 2131 and the barrel shifter B 2132 to obtain the 
"result", and reference numeral 2134 denotes the current 
selection. 

Note, in the third embodiment, delaying of data, 
25 multiplying data by coefficients, and addition of the 

products are performed in order to keep the quality of a 
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reduced image high, however, it is possible to simply 
discard data every four pixels. 

Next, an image sensing operation and a rotation 
operation of an image performed in the image processing 
apparatus ~ac cording to the -t-hi-rd- embodiment of. the., 
present invention having the foregoing configuration are 
explained with reference to flowcharts shown in Figs. 14 
to 17. Note, sequences other than the image sensing 
operation and the rotation operation are not included in 
Figs. 14 to 17. In the third embodiment, SDRAM is used 
as the frame memory, and accordingly, as for operations 
other than the image sensing operation and the rotation 
operation, there is a refreshing operation of the SDRAM, 
and this operation is performed during the vertical 
blanking period when data to be displayed is not 
transferred. 

In Fig. 14, the process starts while waiting for an 
instruction to capture an image. During this process, an 
image which has been stored in the first or second SDRAM 
2201 or 2200, an index image, or a menu, is displayed. 
When a user designates to capture a next image (i.e., to 
scan another film image) using a remote controller, for 
instance, in step S301, then the image sensing system is 
prepared for sensing an image in step S3 02. In the 
preparation, if the backlight is off and it has been a 
considerably long time since an image sensing operation 




has been performed last, for instance, it is turned on 
and the process waits until the backlight is stabilized. 
Then, in step S3 03, units for processing digital signals 
(DSP units) are initialized. This process is performed 

when white~balahce'processing 'and - y correction- are - -- 

necessary for each frame of the film. 

Thereafter, designation of a storage area in a 
frame buffer is performed, as well as the frame buffer 
is initialized, if necessary, in step S304. In this step, 
the mode manager 2210 determines in which of the first 
and second SDRAMs 2201 and 2200, a sensed new image is 
to be stored, and also determines the address area in 
the assigned SDRAM. This process is for properly 
assigning an area of a SDRAM for storing the new image, 
and if the SDRAM only has an enough capacity of storing 
a frame image, the above process is not needed, and in a 
case where an index image is stored in the same SDRAM, 
for instance, the area for storing the new image must be 
designated. In the third embodiment, the memory further 
has an area used for developing an image. Further, in 
the third embodiment, the sensed image is to be stored 
in the first SDRAM 2201, and this condition is set as an 
initial state by the SDRAM controller 2207 via the 
command manager 2209. Further, if the area of the SDRAM 
reserved for the new image is smaller than the area to 
be read out for display, it is necessary to initialize 



an additional area outside of the area where the new 
image is stored with a predetermined value so that the 
outside area of the image on the display screen is 
expressed with a color background. Note that the order 
5 of the processes "o f steps "S3 02 "to" S304~are not - limited - - - 
to above, and the processes may be performed 
simultaneously, and, if any of the processes is not 
needed, it may be skipped. 
Q After initializing respective units for storing the 

fjj 10 new image, the process waits in step S3 05 until 

receiving an flag indicative of the start of the next 
JT frame period. The flag indicating the start of the next 

frame period may be a vertical blanking signal 
5; transferred from the DSP units, or, if the digital data 

H 15 transferred from the DSP units has a predetermined 

format, then the decoded signal of identification 
information specific to the format may be the flag. The 
specific information would be SAV (Start of Active 
Video) in the ITU656 format. After receiving the start 
20 flag of the next frame period, in step S306, the newly 
sensed image is stored in the area of the selected 
memory, i.e., the first SDRAM 2201 designated in step 
S304 in this case. 

When storing the data of the new image, the switch 
25 2o and the three-state buffer 2r are controlled so as to 
temporally store the digital data of the new image from 
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the DSP units in the second dual-port buffer 2204. The 
second dual-port buffer 22 04 stores the data at a 
sampling rate of the DSP units, and transfers the stored 
data to the first SDRAM 2201 at a higher rate than the 

5 sampling rate in step S3 07", "and "the "valid "data* is kept 

stored until the vertical blanking period starts. The 
start of the vertical blanking period may be detected by 
detecting a vertical blanking synchronizing signal 
transferred from the DSP units, or by detecting the EAV 

10 (End of Active Video) in the ITU656 format. The flag 

detector 2212 detects the start of the vertical blanking 
period. The result of the detection is checked by the 
timing observer 2211 and the timing observer 2211 
notifies the state manager 2208 of timing for switching 

15 control sequences. The foregoing processes are performed 
when an image is read by frame (non-interlace scanning) 
using an area sensor such as the CCD. If the CCD reads 
an image by field (interlace scanning) , it is necessary, 
to wait for another vertical blanking period or take 

20 image by two horizontal scan lines. Further, when a line 
sensor is used, scanning position information is added 
to the image data . 

Under this condition, the first SDRAM 22 01 stores 
the image data which is currently displayed; therefore, 

25 the output to the video encoder 2107 is switched by the 
data selector 22 02 from the data outputted from the 
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decoder 2104 to the data outputted from the first SDRAM 
2201 in step S308. By switching the selection by the 
data selector 2202 during the vertical blanking period, 
the change-over of the image does not affect the 
5 displaying o~f "the image. - Note ,~ in - ~the -third -embodiment, 
the synchronizing signals for display are generated on 
the basis of the start timing of frame of the input 
device, thus, the vertical blanking period of the input 
device is especially used. However, if there is a time 

10 lag between input timing and output timing, the time lag 
should be taken into consideration in generating of the 
synchronizing signals . 

While the stored image is being displayed, if a 
user designates to rotate the image, the MPU (not shown) 

15 for controlling the overall operation of the image 

processing apparatus instructs the mode manager 2210 to 
rotate the image in step S3 09 in response to the 
designation by the user. In the example of rotation 
shown in Figs. 29A to 29D, the angle of rotation can be 

20 selected from 90, 180 and 270 degrees. Especially, when 
a 90- or 270-degree rotation is designated, the rotated 
image is reduced. Further, although it is not 
specifically explained in the third embodiment, rotation 
by an arbitrary angle other than the foregoing angles is 

25 possible by using a plurality of frame memories, 

although the structure of an operation unit would be 



-59- 



complicated. Further, mirror image processing (reverse 
of image) should be performed depending upon how a film 
is mounted on the image sensing system. 

If rotation of the image is not designated (NO in 

5 step S309) , and if no instruction for changing the -- - 

displayed image to another image, such as a menu, is 
designated (NO in step S310) , then the process waits for 
an instruction for sensing a new image while displaying 
59 the image which has been stored in the memory. Whereas, 

f\$ 10 if it is designated to display the menu, then the 

t]j process proceeds to perform the designated operation 

l-I (not shown) , and the processing is completed in step 

S310. If the mode manager 2210 receives a control 
M command to rotate the image currently displayed (YES in 

y 15 step S3 09) , then it controls to set a data input /output 

control sequence to be performed in the next frame 
period in the SDRAM controller 2207 via the command 
manager 2209 as well as controls the data selector 2202 
to select the SDRAM which is not currently storing the 
20 valid image data for display (in this case, the second 
SDRAM 22 00) as image memory where the image to be 
processed is to be temporarily stored in step S311. When 
preparation for the rotation operation is ready, the 
process waits the start of the next frame of the image 
25 under display in step S3 12. As the next frame starts, 

storing of the image data starts and is continued until 
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the start of the vertical blanking period is- detected in 
steps S313 and S314. The processes of steps S312 to S314 
are similar to the processes of steps S3 05 to S3 07, thus, 
explanation of them are omitted. 
5 The data -stored i-n -the- memory set in_ step _S311_. (the 

second SDRAM 2200) is for rotation operation, and 
equivalent to the data stored in the memory (the first 
• SDRAM 2201) which is currently displayed. The feature of 
pi the present invention is that the same data of a frame 

10 image is stored in a plurality of memories, which can be 
independently controlled as described above, in a still 
= !? image sensing/displaying apparatus, such as a film 

~ s player, while displaying the frame image stored in one 

Bj memory, the frame image stored in another memory is 

CI 15 processed. In the third embodiment, a case of generating 

and displaying a rotated image by 90 x n (n: ;1, 2, 3) 
degrees with respect to the original image is explained; 
however, it is possible to perform processes other than 
rotation, such as mosaic processing, for instance. 
20 In step S3 15, whether or not it is necessary to 

reduce the image due to rotation of the image by, e.g., 
90 degrees is determined. If the reduction is not 
necessary, then address conversion processing, in which 
the order of the pixels of the image stored in the 
25 second SDRAM 2200 is changed, then the image is stored 

in the development area, is performed in step S320. When 
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the reduction of the image is to be performed, the 
entire frame image is divided into a plurality of blocks, 
and reduction in the horizontal direction, reduction in 
the vertical direction, and the address conversion 
5 processing "for "changing the order- of pixel- positions- are- - ~_ 
performed sequentially for each of the plurality of ' 
blocks. For example, assume that the number of valid 
pixels of the CCD (area sensor) is 640 x 480 pixels, 
P% then it is divided into blocks each having an integer 

5a 10 multiplicati on number of 4x4 pixels for deleting one 

= \& 

of every four pixels, as described with reference to 
^ Figs. 11 to 13. Thus, the image may be divided into 

blocks of 8 x 8 pixels, 16 x 16 pixels, and so on, and 
C3 the number of blocks is 1200 (=40 x 30) in the latter 

Q 15 case. 

=43 In step S316, image data of one block is 

transferred from the second SDRAM 2200 to the first 
dual-port buffer 203. On the image data of one block, 
reduction in the horizontal direction is performed 

20 before rotating the image in step S3 17. The path of the 
data in this process is as follows: the first dual-port 
buffer 2203 the switch 2p the three-state buffer 2r 
— > the barrel shifter 2213 — > the switch 2q — > the switch 
2o the three-state buffer 2r — > the second dual-port 

25 buffer 2204. The state manager 2208 operates the 

switches and three-state buffers at proper time. At this 
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time, the barrel shifter A 2131 and the barrel shifter B 
2132 of the barrel shifter 2213 are controlled so that 
reduction coefficients coff A and coff B as shown in Fig. 
11 are obtained . 
5 After the reduction "in "the "horizontal "direction -is- 

completed, the reduction in the vertical direction is 
performed in the similar manner in step S3 18. The path 
of the data in this process is as follows: the second 
dual-port buffer 2204 — > the switch 2p barrel shifter 

10 2213 the three-state buffer 2s the second dual -port 
buffer 2204. Further, when writing the image to the 
second dual-port buffer 22 04 after reduction, the order 
of pixel positions of the reduced image is changed from 
when reading the image from the second dual -port buffer 

15 22 04, thereby rotation of the image is realized 

simultaneously by block. After all the data stored in 
the second dual-port buffer 2204 has processed, the data 
is transferred to the second SDRAM 2200. At this time, 
addresses of the second SDRAM 2200 are controlled by 

20 block so as to keep the coordinate of the rotated image. 
Although it is not shown in the drawings, as a 
result of the reduction, blank spaces where no image is 
displayed are created on a display screen. To deal with 
these areas, a fixed value is developed and stored in 

25 the SDRAM, selected to store the rotated image, to 

display a color background. The foregoing reduction 
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processes by block are performed for all the blocks of a 
frame image (step S321) . Thereafter, the process waits 
for the start of a vertical blanking period (e.g., frame 
end flag) in step S322, the selection of the SDRAM to be 
5 read for" display is changed frorrr the -f irst -SDRAM— 2201- to - 
the second SDRAM 2200 in step S323, thus the data 
selector 22 02 starts transferring the image stored in 
the second SDRAM 22 00 for display. 

According to the third embodiment as described 
above, after storing image information in an arbitrary 
memory reserved for storing a frame image (first or 
second frame memory 2021 or 2022), while the image 
information is read from the memory and displayed, the 
same image information is copied to another memory also 
reserved for storing a frame image (second or first 
frame memory 2 022 or 2 021) , and the copied image is 
modified. Then, the image on a display screen is changed 
from the image currently displayed to the modified image, 
thereby it is possible to perform a rotation operation 
on the image which is currently displayed, as well as 
the image on the display screen is changed to the 
rotated image without a break in displaying of an image. 
Further, it is possible to develop an image to be 
displayed while displaying an image using a general- 
purpose memory without using video RAM for exclusive use. 
Further, after digitizing input information of a still 



image from the image sensing device 2109, the digitized 
information is stored in the frame memory 2021 (or 2022), 
and read from the frame memory. Then, video signals are 
generated for displaying the still image on the basis of 
5 the read information." ~ . _ 



<Fourth Embodiment> 

In the above third embodiment , as an example of 
m modification of an image, rotation of an image 

Si 10 associated with reduction of the image is explained. In 

7% the reduction of the image associated with rotation of 

the image, fixed value data is written to fill the blank 
areas around the reduced image so as to display a color 
W background. In the fourth embodiment, processing of 

C3 15 improving visual effects of a displayed image by 

iji providing a mask data for a display area outside of a 

valid image (i.e., blank portions in the third 
embodiment) when outputting video signals is explained. 
The mask is a frame for filling blank portions, on 
20 a display screen, around a valid image and, after the 
blank portions are replaced by predetermined frame 
(mask) data, video signals are outputted. Alternatively, 
the blank portions may not be developed, and access to a 
frame memory where the valid image is stored may be 
25 stopped while outputting the mask data. With this 

configuration, it is unnecessary to develop the blank 
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portions, therefore, throughput of developing the image 
is increased. 

Fig. 18 is a block diagram illustrating a 
configuration of a mask processor to be added to the 

5 image processing apparatus"/ "according to~ the- fourth 

embodiment of the present invention. An image processing 
apparatus featuring the fourth embodiment is the same as 
that explained with reference to Fig. 10 in the third 
E3 embodiment, except that a mask manager 2500, a color set 

n| 10 unit 2501, and a switch 2t are added, upstream, to the 

hi video encoder 2107. Note, the basic configuration of the 

image processing apparatus according to the fourth 
embodiment is the same as that explained with reference 
If- to Fig. 8 in the third embodiment. 

f£ 15 Next, the foregoing units are explained below in 

detail. The mask manager 2500 controls the switch 2t to 
disconnect the third dual-port buffer 2205 so as to 
output the mask data from the color set unit 2 501 to the 
video encoder 2107. Further, it also issues start and 

20 stop instructions to develop data in the third dual -port 
buffer 2205 to the buffer manager 2206 in connection 
with the switch control. The color set unit 2 501 is a 
register for developing and storing mask data for 
masking the blank portions, and may output background 

25 information whose initial condition is fixed, or may 

successively develop an arbitrary fixed value (a color 
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factor) outputted from MPU (not shown) . Further, the 
color set unit 2501 may successively develop data so as 
to output an arbitrary pattern, not a solid color, as 
the mask data, and a switch controller for selecting one 
5 of different kinds of 'mask" data ~inay~be~ added.- - Other- - --- 
configuration elements are the same as those explained 
with reference to Fig. 10 , therefore, the explanation of 
them is omitted as well as those units are not shown in 
Fig. 18. 

10 Next, timing control for developing and storing an 

image, and for switching data paths in the image 
processing apparatus according to the fourth embodiment 
of the present invention are explained with reference to 
flowcharts shown in Fig. 19 and 20. The processing shown 

15 in Figs. 19 and 20 are the last half part of a data flow 
on a data path toward the video encoder 2107, and 
performed in parallel to the processing shown in Figs. 
14 to 17. Therefore, the processes for sensing a new 
image and rotating the image are performed in the same 

20 manner as described with reference to the flowcharts 
shown in Figs. 14 to 17, thus, the explanation of the 
processes are omitted. Below, processing of providing 
mask data is explained with reference to the flowcharts 
in Figs. 19 and 20. 

25 The fourth embodiment is applicable to any 

processing for masking a blank portion; however, an 
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image which is reduced associated with rotation as shown 
in Fig. 29B or 29D is explained as an example. Below, a 
stage for output ting an reduced and rotated image for 
display is explained. The image data is developed little 

5 by little at high speed' in "the "third "dual"- port" "buff er 

2205, while an output speed of the image data from the 
buffer 22 05 is controlled to the speed desired by the 
video encoder 2107. In this stage, since the portion on 
the left of the reduced image is subjected to a masking 

10 operation, and the masking operation starts from a step 
of detecting a start position of masking operation in 
step S510. Note that the start position of the masking 
operation is generally arbitrary, and the cases as shown 
in Figs. 29B and 29D are simply examples. Thus, when 

15 valid pixels are to exist in the leftmost position of a 
display screen (i.e., the image is arranged at the 
leftmost position, and the masking operation should not 
be performed in the left portion) , by using a parameter, 
such as a register for setting an arbitrary masking 

20 start point, and controlling it, it is possible to 

control the start mask point, e.g., not to mask the left 
portion. 

When the start point of an area, subjected to the 
masking operation, on the left of the valid image 
25 (referred to as "left mask start point") is detected 

(YES in step S510) , the mask manager 250 0 controls the 
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switch 2t so as to output mask data set in the color set 
unit 2501, then controls the buffer manager 2206 so as 
to stop outputting image data from the third dual-port 
buffer 22 05 in step S511. The mask data is output ted 

5 until a mask end po~irit is "detected' in "step" S512\ - If - one - 

horizontal line has been scanned without the mask end 
point being detected (YES in step S513), the process 
proceeds to step S518. Note that information on 
p coordinates of the start points of areas subjected to 

%l 10 the masking operation on the right and left of the valid 

\l- image may be fixed by hardware or may be set by the MPU 

J* (not shown) . 

?3! When the mask end point is detected (YES in step 

S512), then the image data starts being outputted to the 
H_ 15 video encoder 2107 (i.e., the switch 2t is changed to 

13 the third dual -port buffer 22 05) , and the image data, 

outputted from the first or second SDRAM 2201 or 2200 
and developed in the third dual-port buffer 2205, is 
outputted by bit from the third dual-port buffer 2205 in 
20 step S514. The image data output (step S514) is 

performed until a start point of an area, subjected to 
the masking operation, on the right of the image 
(referred to as "right mask start point" hereinafter) is 
detected in step S515. In step S519, whether or not the 
25 image data is outputted until the end of a horizontal 
scanning line is detected without the right mask start 
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position being detected is determined, and if it is (YES 

in step S519), then the process proceeds to step S518. 

If the right mask start point is detected in step S515, 

then in steps S516 and S517, the mask data is controlled 
5 to be outputted" to" the' video- encoder 2-107 -until -the- end- . 

of one horizontal scanning line. 

The aforesaid processes are continued unless 

another image is designated to be displayed in step S518 . 
Q Further, the determination of changing images to be 

nj 10 displayed, and actual change-over of displayed images 

i!j (change-over of statuses of SDRAMs) are performed during 

the vertical blanking period of the synchronizing period 
J* of the video signal. With the masking processing as 

5*5 described above, an area of a SDRAM corresponding to the 

H 15 blank portion where no valid image data is stored does 

h3 not need to be developed, thereby throughput for 

developing and displaying an image is improved. 

According to the fourth embodiment as described 

above, in addition to the effects obtained in the third 
20 embodiment, it is possible to improve visual effects of 

a displayed image by providing mask data for a display 

area outside of a valid image area (i.e., blank portions 

in the third embodiment) when outputting video signals. 

Further, the blank portions is not developed, and access 
25 to a frame memory (2021 or 2022)) where the valid image 

is stored is not stopped while outputting the mask data. 
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With this configuration, it is unnecessary to develop 
the blank portions, therefore, throughput of developing 
the image is increased. 

In the image processing apparatuses described in 
5 the third and fourth embodiment's" as "described- above, -- - 
image data stored in frame memory is read out and video 
signals are generated on the basis of the read image 
data for displaying an image. However, the present 
p invention is not limited to this, and read out data may 

£j 10 be used in various ways, such as outputting the data to 

rlj an external device, for instance. As an example of a 

i2 configuration of the image processing apparatus in a 

^ case where image data is transferred to an external 

j** device, an input interface 3221, a controller 3222, 

M 15 frame memory 3223, and an output interface 2224, as 

€1 shown in Fig. 21, are included. 

Further, the present invention can be applied to a 
system constituted by a plurality of devices or to an 
apparatus comprising a single device. 
20 Further, the object of the present invention can 

also be achieved by providing a storage medium storing 
program codes for performing the aforesaid processes to 
a computer system or apparatus (e.g., a personal 
computer) , reading the program codes, by a CPU or MPU of 
25 the computer system or apparatus, from the storage 
medium, then executing the program. 
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In this case, the program codes read from the 
storage medium realize the functions according to the 
embodiments, and the storage medium storing the program 
codes constitutes the invention. 

Further, the storage medium, such as ~a~ floppy" disk," 
a hard disk, an optical disk, a magneto-optical disk, 
CD-ROM, CD-R, a magnetic tape, a non-volatile type 
memory card, and ROM can be used for providing the 
program codes . 

Furthermore, besides the aforesaid functions 
according to the above embodiments are realized by 
executing the program codes which are read by a computer, 
the present invention includes a case where an OS 
(operating system) or the like working on the computer 
performs a part or the entire processes in accordance 
with designations of the program codes and realize 
functions according to the above embodiments. 

Furthermore, the present invention also includes a 
case where, after the program codes read from the 
storage medium are written in a function expansion card 
which is inserted into the computer or in a memory 
provided in a function expansion unit which is connected 
to the computer, CPU or the like contained in the 
function expansion card or unit performs a part or the 
entire process in accordance with designations of the 



program codes and realizes functions of the above 
embodiments . 

The present invention is not limited to the above 
embodiments and various changes and modifications can be 
made within the spirit and scope of "the "present " --- - - -- 

invention. Therefore to apprise the public of the scope 
of the present invention, the following claims are made. 
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